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Military  aircraft  missions  are  all  multidimensional  in 
nature.  This  means  that  every  mission  can  be  divided  into 
usually  one  overall  goal  or  purpose  (i.e.,  destroy  the 
target,  deliver  the  supplies,  rescue  the  survivors,  etc.), 
with  several  subgoals  (safety,  minimize  susceptibility, 
timeliness,  etc.).  Since  missions  are  multidimensional,  the 
operator  effort  in  the  form  of  mental  and  physical  action 
(performance)  becomes  multidimensional.  The  multidimentional 
nature  of  skilled  aircrew  performance,  in  turn,  reguires 
that  several  criteria,  all  of  which  are  relevant  for  a 
particular  activity,  be  defined  and  used  [ Ref •  1 ]. 

The  unique  situation  of  an  aircrew  flying  an  aircraft 
for  a  specific  mission  and  the  necessary  determination  of 
subcriteria  for  evaluating  accomplishment  of  that  mission 
requires  further  research  of  an  analytical  and  empirical 
nature.  The  relationship  among  altitute,  airspreed, 
operator  activity,  and  the  hundreds  of  other  system  vari- 
ables that  comprise  the  total  system  must  be  compared  to 
mission  success  in  quantifiable  terms. 

This  study  is  an  effort  to  improve  acquisition  of 
training  performance  information  in  affortable  ways  on 
behalf  of  the  Hellenic  Air  Force  (HAF) .  Thus,  it  is  divided 
in:  (1)  The  principles  of  human  performance,  (2)  Definition 
of  the  criteria  and  their  measurement,  (3)  Systematic  defi- 
nition of  performance  measure  appropriate  to  combat- training 
needs,  (4)  Definition  of  a  cost  effective  measurement  system 
usable  in  combat-crew  training  environments  to  acquire  and 
process  needed  training  information. 

The  method  that  was  used  was  a  search  of  the  available 
materials  found  in  the  NPS  library.  Most  of  the  data  found 
was  based  on  DSAF  studies. 
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I.  INTRODUCTION 

The  Hellenic  Air  Force  (HAF)  is  interested  in  optimizing 
its  combat  readiness,  and  consequently  the  combat  readiness 
of  its  combat  pilots.  Greece  has  common  borders  with 
hostile  countries  and  is  constantly  threatened  with  possible 
invasion.  There  are  continuous  violations  of  the  aerial  and 
sea  space  of  Greece  by  neighboring  countries'  airplanes  and 
ships  respectively.  He  (the  Greeks)  must  offset  our 
numerical  inferiority  with  technology  expressed  in  higher 
skilled  combat  pilots.  It  is  imperative  to  increase  to  the 
utmost  the  combat  readiness  and  effectiveness  of  the  HAF. 
In  this  way  we  shall  be  a  strong  power  and  our  enemies  will 
never  try  to  invade  our  country.  Thus,  we  shall  remain  in 
our  purpose,  that  is,  united  and  uninvaded  country  and  most 
important,  because  of  our  power  we'll  keep  the  peace. 

Today  the  need  for  combat  effectiveness  is  more  pressing 
than  ever  before.  We  live  in  a  period  of  rapid  and 
continuous  change  in  technology  and  consequently  in  our 
tactics.  The  new  generation  of  combat  aircraft  require  more 
training  because  of  the  complexity  of  the  new  systems. 
Eecause  of  these  technological  changes  it  is  required  that 
new  changes  in  air  tactics  must  be  established.  Thus, 
because  of  the  above  continuous  changes,  it  is  required  that 
new  criteria  for  combat  readiness  of  combat  pilots  must  be 
established  (created).  Of  course,  HAF  has  established 
criteria  for  its  combat  pilots  as  a  result  of  the  entry  of 
the  present  generation  of  combat  aircraft.  However,  it  may 
be  necessary  to  see  if  there  are  any  critical  points  that 
need  to  be  corrected  or  changed.  For  these  reasons  this 
study  has  been  conducted  in  which  the  US  Air  Force  was  used 
as  a  model  and  a  later  study  will  be  necessary  to  adapt 
these  models  to  the  Greek  reality. 
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Thus,  this  study  will  define  combat  readiness,  the 
criteria  and  their  considerations  and  human  performance,  in 
relation  to  the  combat  readiness  of  combat  pilots. 

A.   CCMBAT  READINESS 

But  what  is  combat  readiness?  In  order  to  answer  this 
guestion,  we  first  must  define  the  word  readiness,  secondly 
combat  readiness,  and  third  readiness  categories. 

Readiness  means  the  guality  of  being  ready  to  act, 
respond,  comply,  etc;  Also  readiness  is  a  state  or  fact  of 
being  ready,  or  prepared;  as  to  have  everything  in  readiness 
for  a  sudden  departure  [Ref.  2].  A  crew  or  crew  member  is 
combat  ready  when  it  is  certified  as  ready  for  combat 
[Ref.  3]. 

1 .   Readiness  Categories : 

Combat   readiness  is   expressed   with  the   following 
standard  categories  and  meanings  [Ref.  4]. 

a.  Fully  Combat  Ready. 

A  unit  fully  capable  of  performing  the 
mission  (s)  for  which  it  is  organized  or  designed. 

b.  Substantially  Combat  Ready. 

A  unit  capable  of  performing  the  mission  (s)  for 
which  it  is  organized  or  designed,  but  having  minor  defi- 
ciencies which  could  reduce  its  effectiveness  or  its  ability 
to  conduct  sustained  operations. 

c.  Marginally  Combat  Ready. 

A  unit  with  major  deficiencies  of  such  magnitude 
as  to  severely  limit  its  capability  to  perform  the 
mission  (s)    for  which  it  is   organized   or  designed,    but 
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capable  of  conducting   limited  operations   for   a   limited 
period. 

d.   Not  Combat  Ready. 

A  unit   not  capable  of  perf oming   the  mission  (s) 
for  which  it  is  organized  or  designed. 

Definitions  of  readiness  and  combat  readiness 
were  given  above.  A  classification  of  combat  readiness  into 
four  categories  was  made  in  an  effort  to  specialize  and 
highlight  it.  In  order  to  define  and  understand  combat 
readiness  better  we  have  to  define  criteria  and  their 
considerations,  that  is  to  say,  definition  and  purpose  of 
criteria,  the  categories  of  criteria  and  characteristics  of 
good  criteria. 

E.   CRITERIA  AND  CONSIDERATIONS 

1 .   Definition  and  Purpose  of  Criteria 

Criteria  are  standards,  rules,  or  tests  by  which 
measures  of  system  behavior  are  evaluated  in  terms  of 
success  or  failure,  or  to  some  degree  of  failure.  The 
purpose  of  human  performance  criteria  is  to  provide 
standards  or  baselines  for  evaluating  the  success  or 
failure,  goodness  or  badness,  or  usefulness  of  human 
performance  [Ref.  5]  and  [Ref.  6]. 

Criteria  should  not  only  define  the  unique  manner  in 
which  the  operator  should  perform  a  task,  but  should  define 
the  performance  objectives  of  the  entire  man-machine  system 
[Ref.  7]  and  [Ref.  8  ]- 

2  •   Categories  of  Criteria 

The  classification   of  criteria  can   be  accomplished 

from  a  measurement  standpoint;  beginning  with  the  smallest 

known  entity  and  ending  with  the  "ultimate"   quantity  that 

may  exist.   Several  categories  identified  are  listed  below: 
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(1)  Parametric  referent  or  standard  of  performance 
which  is  sought  to  be  met  by  the  operator  or  system. 
Example:  maintain  1000  feet  of  altitude  [Ref.  7]  and 
[Bef.  9]- 

(2)  Parametric  limit  about  the  parametric  standard 
within  which  the  operator  or  system  is  required,  or 
seeks,  to  remain.  Example:  maintain  plus  or  minus 
100  feet  while  at  1000  feet  altitude  [Bef.  7]  and 
[Bef.  9]. 

(3)  System  component  criteria  which  distinguishes 
the  relationship  between  system  components  and  system 
output.  Example:  "  least  effort  "  measured  from 
the  pilot  in  relation  to  maintaining  altitude 
[Bef.  10]. 

(4)  Test  criterion  used  to  evaluate  overall  human 
ability,  usually  expressed  as  a  single  overall 
measure.  Example:  subjective  judgement  of  instructor 
for  a  student  as  to   "pass"  or  "fail". 

(5)  Ultimate  criteria  are  multidimensional  in  nature 
and  represent  the  complete  desired  end  result  of  a 
system.  This  category  of  criteria  is  impossible  to 
guantify  due  to  the  multidimensional  nature  of  the 
system's  purpose,  and  hence,  is  a  theoretical  entity 
that  must  be  approximated.  Example:  Any  aircraft's 
mission  [Bef.  1]  and  [Bef.  11]. 

It  may  be  noted  that  all  five  categories  of  criteria 
can  be  either  quantified  or  approximated  in  some  manner, 
with  decreasing  accuracy  as  the  ultimate  criteria  level  is 
reached. 

Obtaining  direct  measures  of  the  ultimate  criteria 
for  a  complex  system  is  seldom  feasible.  This  is  particu- 
larly true  in  military  systems  where  such  criteria  would  be 
expressed  in  terms  of  combat  effectiveness  or  effectiveness 
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in  preventing  a  potential  aggressor  from  starting  a  conflict 
[Eef.  1]. 

Therefore,  it  becomes  apparent  that  we  must  select 
intermmediate  criteria  (types  (1)  through  (U)  above)  in 
evaluating  skilled  operator  behavior. 

3.   Characteristics  of  Good  Criteria 

Using  actual  criteria  as  approximations  of  the  ulti- 
mate criteria  can  be  accomplished  by  several  methods  that 
will  be  presented  in  the  following  paragraphs  . 

Although  there  is  no  certain  method  that  will  lead 
to  the  specification  of  good  criteria,  there  are  seme 
considerations  that  can  be  taken  into  account  which  are 
discussed  below: 

(1)  A   good   criterion  is  both   reliable  and   relevant 
[Bef.  1]  and  [  Ref .  10]. 

(2)  Criteria  must  be  comprehensive  in  that  the  utility  of 
the  individual  being  evaluated  is  unambiguously  reflected. 

(3)  Criteria  should  possess   selectivity   and  have  ready 
applicability  [Eef.  10]. 

Finally,  since  readiness  and  criteria  are  related  to 
human  performance,  it  is  necessary  to  define  human 
performance. 

C.   HUMAH  PEBFOBflANCE 

Much  of  the  data  used  in  human  factors  consist  of 
measures  of  some  aspect  (s)  of  human  performance.  Human 
performance  in  the  context  of  systems  often  boils  down  to 
consideration  of  how  fast  people  can  perform  their  functions 
and  how  accurately  they  can  perform  them.  How  well  functions 
are  performed  leads  directly  to  considerations  of  human 
errors  [Eef.  12]. 
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D.   THE  PROBLEM 

Thus,  the  problem  is  the  necessity  of  optimizing  the 
combat  readiness  of  HAF's  combat  pilots,  because  of  the 
threatened  invasion  by  neighboring  countries.  So,  we  have 
to  identify  the  necessary  variables  which  will  assist  the 
planner  (commander)  to  be  certain  that  his  combat  pilots 
will  meet  the  desired  level  of  combat  effectiveness  in  terms 
of  pilot  capabilities- 
It  can  be  suggested  that  for  HAF  the  first  requirement 
is  to  continuously  optimize  combat  readiness.  To  accomplish 
this  objective  it  is  essential  that  combat  pilots  be  at  an 
acceptable  level  of  combat  readiness.  Determination  of 
combat  readiness  requires  definition  of  concept,  specifica- 
tion of  criteria  and  a  definition  of  human  performance. 

In  the  chapters  to  follow  an  attempt  will  be  made  to 
define  and  discuss  the  following:  (a)  the  principles  of 
human  performance  and  (b)  the  criteria  and  their  measure- 
ment. Finally,  chapters  IV  and  V  will  examine  combat  ready 
crew  performance  and  derivation  of  combat-crew  performance 
measurement  techniques. 
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II-  PRINCIPLES  OF  BO MAN  PERFORMANCE 

A.   HOHAN  IERFOBMANCE  REQOIREMENTS 

We  can  best  determine  performance  requirements  only 
after  we  have  a  good  set  of  allocated  functions.  In  the 
absence  of  function  allocation  we  will  have  a  difficulty 
identifying  and  dealing  effectively  with  human  performance 
requirements  because  we  do  not  know  what  functions  people 
will  be  expected  to  perform. 

Once  we  have  established  a  human  performance  requirement 
we  should  develop  a  way  of  measuring  it.  If  the  requirement 
relates  to  accuracy,  then  there  must  be  a  meaningful  way  to 
measure  errors.  If  the  requirement  concerns  manual 
processing  time,  there  must  be  a  meaningful  way  to  measure 
it.  For  example,  time  per  customer  contact,  number  of  items 
produced  per  hour,  and  average  keystrokes  per  day. 

Usually  the  problem  is  deciding  what  measure  gives  the 
best  indication  of  performance.  It  has  been  suggested  that 
with  a  little  imagination,  any  human  performance  can  be 
meaningfully  measured  [Ref.  13].  As  a  minimum,  human 
performance  requirements  should  include  statements 
concerning  errors,  manual  processing  time,  training  time 
necessary  to  ensure  the  minimum  skills,  and  job 
satisfaction.  If  we  do  not  clearly  state  the  requirements 
from  the  beginning  we  cannot  expect  human  performance 
considerations  to  be  taken  seriously.  And  when  the  system 
is  operational,  people  will  be  left  to  perform  as  best  they 
can  without  adequate  provision  for  ensuring  an  acceptable 
level  of  human  performance  [Ref.  14]. 

Other  human  performance  requirements  could  be  associated 
with   training  time   (e.g.,   total   time   to  train   clerical 
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personnel  to  perform  the  basic  activities  should  not  exceed 
three  weeks) .  Another  human  performance  requirement  could 
relate  to  job  satisfaction  (e.g.,  after  performing  an 
activity  for  six  months  employees  should  respond  in  a  posi- 
tive way  to  their  work,  as  measured  by  a  questionnaire) . 

The  identification  of  human  perfomance  requirements  is  a 
prerequisite  step  for  a  system's  development  process.  One 
of  the  steps  of  a  system's  development  process  is  the  task 
analysis.  Task  analysis  is  intended  to  match  the  work  to  be 
done  with  the  kinds  cf  people  who  will  do  it.  The  process 
has  four  main  parts: 

(1)  Determination  of  a  system  structure  that  gives  the 
designer  an  overall  view  or  objective  for  the  analysis. 

(2)  Identification  of  tasks  has  essentially  nine  primary 
considerations  associated  with  it.  These  include 
determining  existing  knowledge  and  skills,  deriving  skill 
level  categories,  identifying  outputs  and  inputs,  deriving 
lower  level  activities,  ensuring  that  activities  are  mutu- 
ally exclusive  and  exhaustive,  and  matching  active 
complexities  with  previously  determined  skill  levels.  The 
process  may  be  repeated  any  number  of  times  until  each 
activity  is  assigned  a  single  skill  level.  Secondary 
considerations  in  the  identification  of  tasks  include 
meeting  the  system  structure  objective  just  discussed,  as 
well  as  meeting  a  full-advantage  objective.  A  designer 
attempts  to  develop  tasks  that  will  ultimately  take  full 
advantage  of  the  user  work  force.  This  is  difficult  to 
quantify,  but  during  the  analysis  process,  most  designers 
gain  a  feel  for  what  is  meant  by  taking  full  advantage  of 
the  skills  available  in  their  user  population  and  this 
should  be  reflected  as  the  tasks  are  being  identified. 

(3)  Description  of  each  task  and  organization  of  all  tasks 
into  a  flowchart  that  will  accomodate  the  variety  of 
different  transactions  a  new  system  must  accomplish. 
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(4)  Identification  of  work  modules  that  synthesizes  the 
tasks  previously  identified  into  manageable  modules  of 
work.  A  human  work  module  is  a  set  of  tasks  that  a  user 
accomplishes  as  a  part  or  all  of  his  or  her  job  [Ref.  14]. 
It  is  a  basic  unit  work.  Usually,  one  or  more  work  modules 
are   combined   to  form   a   job. 

Having  systematically  derived  work  modules  assists  in 
the  design  of  interfaces,  and  the  preparation  of  facilitator 
materials,  such  as  instructions,  performance  aids  and 
training. 

B.       FACILITATING    HUMAN    PERFORMANCE 

Once  good  work  modules  are  developed, many  types  of 
materials  can  be  developed  to  help  ensure  an  acceptable 
level   of      human    performance.  All    of      these   materials      are 

based  on  the  task  analysis  results  and  are  usually  prepared 
to   support    the    work    module   instructions. 

Once  the  designer  identifies  tasks  and  determines  work 
modules,  he  must  identify  the  specific  skills  and  knowledge 
required  for  each  work  module.  This  amounts  to  stating  his 
assumptions  of  the  precise  qualifications  of  the  person  to 
perform    the   work   module. 

As  the  work  modules  are  designed,  the  designer  should 
have    a      set   of      user    characteristics      in  mind.  The   design 

should  clearly  envision  the  person  performing  the  work. 
When  this  information  is  written  down,  it  is  called  a 
Statement   of   Minimum    Qualifications    or   SMQ    [Ref.     14]. 
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The  SMQ  is  a  detailed  description  of  minimum  acceptable 
qualifications  in  terms  of  skills  and  knowledge  required  to 
efficiently  and  effectively  perform  the  work  outlined  by  the 
work  module. 

Once  the  designer  has  in  mind  the  potential  user  and  has 
designed  the  work  modules,  then  he  can  proceed  with 
developing  instructions,  performance  aids  and  training. 

It  is  vitally  important  that  human  performance 
considerations  begin  at  least  by  the  time  functions  are 
being  allocated.  Having  a  good  set  of  human  performance 
requirements  is  also  critical.  A  systematic  and  detailed 
analysis  of  the  tasks  to  be  performed  is  also  very 
important. 

If  design  considerations  are  well  done,  then  a  strong 
foundation  is  established  for  developing  interfaces  and 
facilitators,  including  Human/Computer  dialog  structures, 
instructions,  performance  aids,  training  materials. 

In  this  chapter  the  principles  of  human 
performance,  namely  (a)  Human  Performance  requirements 
and  (b)  Facilitating  human  performance  have  been  discussed. 

In  order  to  measure  one  person's  attitude,  aptitude  and 
performance  we  may  use  criteria.  Thus,  in  the  next  chapter 
we  shall  expand  on  the  concept  of  criteria  and  their 
measurement. 
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III.  CRITERIA  AND  THEIR  ME  A  SO  RM  MI 

As  suggested  earlier,  a  criterion  is  a  standard  which 
may  be  used  to  evaluate  a  person's  attitude,  aptitude,  and 
performance.  Since  personnel  selection  and  training  are 
important  factors  for  HAF  to  optimize  combat  readiness  of 
its  ccmbat  pilots,  we  will  be  concerned  with  the  use  of 
criteria  for  personnel  selection  and  training  purposes. 

Selection  criteria  are  described  by  the  degree  of  corre- 
lation between  selection  test  scores  and  performance 
measures  (in  real-world  situations) ,  which  represent  the 
degree  of  a  pilot's  success  in  performing  his  job. 

Training  criteria  refer  tc  measures  utilized  in  evalu- 
ating the  effectiveness  of  a  training  program  (i.e., the 
measures  that  express  the  degree  to  which  the  attainment  of 
the  behaviorial  objectives  of  the  training  program  have  been 
met)  [Ref.  15]. 

A.   OSE  CF  CRITERIA  IN  PERSONNEL  SELECTION  AND  TRAINING 

Personnel  selection  serves  to  predict  a  person's 
suitability  for  a  job,  whereas  the  purpose  of  personnel 
training  is  to  derive  a  predetermined  work  standard,  or 
other  criterion,  in  the  shortest  possible  training  time. 
Both  purposes  can  be  achieved  by  analyzing  and  quantifying 
the  content  and  skills  associated  with  the  job.  The  valida- 
tion of  each  process  can  be  evaluated  as  follows  : 

1.  The  validity  of  a  personnel  selection  system  can  be 
evaluated  by  the  degree  of  the  relationship,  or  correlation, 
between  test  scores  against  the  criteria  measures. 

2.  The  validity  of  a  training  method  can  be  evaluated  by 
comparing  a  variety  of  different  training  methods  with  a  set 
of  the  same  criteria   for  all  cases. 
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The  validity  of  selection  and  training  programs  alike  is 
affected  by  the  following  factors  : 

1.  The  degree  to  which  personnel  selection  tests  or 
personnel  training  procedures  simulate  real-world  criteria. 

2.  The  validity  of  personnel  selection  tests  and 
training  under  study. 

3.  Validity  of  the  criteria  considered,  true  and  not 
merely  correlational. 

4.  Relevance  of  the  criteria  considered  for  (a)  the  job 
studied,  and  (b)  the  test  batteries  adopted  for  selection  or 
training  purposes. 

5.  The  stage  (initial,  ultimate,  or  rate  of  learning)  at 
which  criteria  are  measured  may  affect  validities  signifi- 
cantly. Typically,  criteria  measures  at  the  early  stages  of 
the  acquisition  of  skill  exhibit  higher  validity  coeffi- 
cients than  those  we  find  when  the  measures  are  correlated 
with   terminal  performance  [ Ref .  16]. 

6.  Because  job  performance  is  multidimensional,  taking 
only  one  criterion  as  an  index  of  job  performance  may  result 
in  artificially  high  or  low  validity  coefficients. 
Therefore,  a  multidimensional  approach  is  essential  in  the 
measurement  of  job  performance  and  criteria  utilization  in 
personnel  selection  and  training. 

7.  Evidence  indicates  that  criterion  and  test  measures 
are  typically  nonlinear  [Ref.  17]  [Ref.  18]  and  [Ref.  19]. 
Since,  moreover,  assuming  their  linearity  increases  the 
error  variance  of  the  validity  coefficient,  the  validity  of 
the  assumption  must  be  tested  before  linearity  is  assumed. 

8.  Fluctuations  in  an  individual's  physiological  and 
psychological  health  condition,  need  of  achievement  motiva- 
tion, and  so  on,  affect  the  validity  coefficient  of 
personnel  selection  programs  as  well  as  training  programs. 
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In  summary,  it  is  hardly  possible  to  overstress  the 
importance  of  a  sound  criterion  measure  against  which 
personnel  selection  and  training  programs  can  be  evaluated 
effectively.  In  order  to  achieve  this  objective,  emphasis 
must  te  placed  at  the  outset  on  the  development  of  such 
measures.  Without  sound  criteria  the  true  validity  of 
personnel  selection  and  training  cannot  be  assessed 
properly. 

B.   TIEES  CF  CRITERIA. 

Criteria  for  the  purposes  of  personel  selection  and 
training  lie  on  a  continuum  scale,  which  has  two  interactive 
dimensions  :  objectivity-subjectivity  and  employee-employer 
satisfaction.  Some  criteria  are  entirely  subjective;  others 
are  entirely  objective.  Most,  however,  involve  a  mixture  of 
the  two.  In  other  words  we  distinguish  two  types  of 
criteria:  (1)  Those  describing  the  satisfaction  of  the 
empoyee's  needs  and  (2)  those  describing  the  satisfaction  of 
empoyers  in  their  empcyees. 

The  ideal  criterion,  although  rarely  found  in  industry, 
is  the  one  that  reflects  the  satisfaction  of  employee's 
needs  as  well  as  that  of  empoyers*.  A  criterion  like  this 
is  used  more  frequently  for  personnel  guidance  purposes  than 
in  personnel  selection  and  training  [Ref.  20]. 

It  is  essential  to  aim  at  using  the  most  appropriate 
criterion  for  a  specific  job,  rather  than  the  most 
convenient  criterion,  but  it  is  not  always  feasible  to 
assess  the  former  or  to  obtain  data  on  it.  Thus,  the 
analyst  is  frequently  faced  with  the  practical  issue  of 
using  the  most  appropriate  criterion  which  data  are 
available  or  can  easily  be  collected. 

The  reason  for  using  a  criterion  varies  since  occupa- 
tional  jobs  are   different;    consequently,    a  variety  of 
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criterion  measures  have  been  developed  and  used  during  the 
last  century,  each  with  its  characteristic  weaknesses  and 
strengths.  However,  any  criterion  could  be  the  most  appro- 
priate for  a  specific  job,  depending  on  circumstances  and 
purpose.  The  majority  of  the  criteria  relevant  to  today's 
occupational  circumstances  and  competence  are  listed  in 
Table  1.  • 

For  seme  of  the  criteria  mentioned  in  Table  1,  objective 
data  can  be  gathered  (e.g.,  length  of  employee's  service), 
ethers  require  subjective  assessment  (e.g.,  rating  or 
ranking  by  supervisor,  subordinates,  members  of  equal 
status,  and  the  employee  himself) .  For  some  criteria  (e. g. , 
quality  of  output)  both  subjective  and  objective  techniques 
can  be  utilized.  Many  of  these  can  be  related  to  training 
progress  at  the  following  stages:  (a)  immediate,  at  the 
beginning  of  training;  (b)  intermediate,  at  the  end  of 
training;  (c)  ultimate,  in  real-world  operations,  after 
completion  of  training. 

C.   CRITERIA  MEASURES 

There  are  basically  three  types  of  methods  by  means  of 
which  criteria  data  can  be  collected  : 

1.  Rating  or  ranking. 

2.  Counting  (e.g.,    the  number  of  items  produced,   the 
number  of  accidents,  the  duration  of  service)  . 

3.  Establishing  work  standards  £Ref.  21]. 

In   this  chapter   we  have  seen    the   criteria  and   their 

measurement  and   specifically    (a)    Use   of   criteria  in 

personnel  selection  and  training,   (b)  Types  of  criteria  and 

(c)   Criteria  measures.    Next  we  shall  introduce  the  combat 

ready  crew  performance  measurement. 
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TABLE  1 

Tyres  of  Criteria  Utilized  for  Personnel  Selection  S 

Training 

Criteria    reflecting    employees'    competence 
A.      Quality   of    performance 
T.    Quality    cf   output 

2.  Spoiled  work: 

(a)  amount 

(b)  cost 

3.  Accidents: 
(a)  number 

Jb)  cost  (financial  and  human) 

4.  Breakages  (tools/etc.)  : 

(a)  number 

(b)  cost 

5.  Mistakes  in  operation: 
number 


Jbj 
ar 


6.  variability  in  performance 

*7.  Rate  of  advancement 

*8.  Standard  trade  examinations 

*9.  Training: 


(a)  cos- 

J1) 

(b)  duration 


(i)  to  employer,   (ii)  to  employees 
^b)  duration 
B.  Quantity  of  performance 


1.  Quantity  of  output 

2.  Earnings  on  a  commission  basis 

3.  Earned  bonus 

4.  Peak  performance 

5.  Lowest  performance 
*6.  Rate  of  advancement 

*7.  Standard  trade  examinations 
*8.  Training  : 

(a)  cost 

(i)  to  employer,  (ii)  to  employees 

(b)  duration 

II.  Criteria   reflecting    employees'    circumstances 

1.  Length   of   service 

2.  Labor   turnover 

3.  Lateness    (categorized   according   to   how    late) : 

(a)  numbers 

(b)  cost 

4.  Absenteeism  : 

(a)  certified 

(i)  number,  (ii)  cost 

(b)  uncertified 

(i)    number,     (ii)    cost 

III.  Criteria   reflecting   employee's    satisfaction 
from   job 

1.  Rating    of   employee's   liking  for   his  present    job 

2.  Rating    of   employee's   satisfaction    with 

job  content   and   desire   for    job   enlargement 
or   job    simplification 


*    Starred   items    may    reflect   quality   and/or   quantity    of 
performance.    Since    their   relationship   to   performance 
is   indirect,    they   are   less    valuable   and   important 
than   the    others. 
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IV.  INTRO DOCT ION  TO  COMBAT  READY  CRES  PERFORMANCE 

MEASUREMENT 

Recently,  the  evaluation  of  aircrew  proficiency  in 
skills  associated  with  advanced  flying  training  was  prima- 
rily based  upon  the  subjective  judgement  of  instructor 
pilots.  In  addition  these  evaluations  are  supplemented  by 
the  mere  or  less  objective  scores  of  gunnery  and  bombing.  In 
certain  areas,  such  as  air  combat  maneuvering  and 
combat-readiness  determination  the  evaluations  are  totally 
objective. 

Although,  efforts  have  been  made  to  measure  objectively 
the  behavioral  skills  in  the  operational  and/or  crew 
training  setting  in  an  economical  way,  these  efforts  have 
not  yet  produced  any  positive  results. 

A  great  percentage  of  the  HAF's  budget  is  absorbed  by 
the  aircrew  training  costs,  therefore,  this  area  has  become 
of  fiscal  concern.  Due  to  the  projected  increase  in  costs 
cf  operating  the  newer  weapons  systems  over  the  present 
generation  of  combat  aircraft  the  HAF's  interest  in  the 
problem  has  intensified  in  recent  years.  Studies  to  minimize 
costs  have  shown  that  this  can  be  achieved  by  reducing  the 
training  flying  hours  and  transfer  them  (when/where  this  is 
possible)  to  lower  cost  devices,  e.g.  simulators. 

The  HAF  is  trying  to  adapt  modern  Systems  Approach  to 
Training  techniques  to  aircrew  training  programs.  Existent 
in  the  Systems  Approach  to  Training  concept  is  student 
advancement  on  individual  proficiency  rather  than  course 
length.  The  traditional  subjective  methods  of  evaluation  may 
be  proved  insufficient  for  the  more  complex  aircrew  training 
objectives. 
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A.   OBJECTIVES 

Necessity  of  improving  training  performance  information 
of  HAF's  pilots,  directed  this  study  to  the  following 
objective  based  on  experiments  and  studies  of  OSAF 
[Ref.    22]    : 

Goal:  Systematic  definition  of  performance  measures 
appropriate        to  combat-crew  training  needs. 

Performance  measures  will  include  formal  statements  of 
methods  of  measuring  flight  crew  performance  used 
during  and  at  the  end  of  combat-crew  training,  and,  new 
measures  meaningful  to  combat-crew  training  and  useful 
as    tools   for   training   research. 

This  study  thus  is  an  effort  to  describe  usable  measure- 
ment tools  for  utilization  in  combat-crew  training 
research. 

1 .      Instructional  System   Development. 

Research  studies  directed  toward  performance 
measurement  in  combat-crew  training  are  highly  relevant 
today  in  view  of  USAF  policy  to  employ  a  systems  approach  to 
flying  training  problems.  The  model  for  Instructional 
System  Development  (cf.,  Dept.  of  Air  Force,  1970)  contains 
the    following   basic    steps: 

1.  Analyze  system  requirements. 

2.  Define  education  or  training  requirements. 

3.  Develop  objectives  and  tests. 

4.  Plan,  develop,  and  validate  instruction. 

5.  Conduct  and  evaluate  instruction. 

In  support   cf   instructional   system   development, 

measures  and   a  measurement   system   are   necessary   to: 

(1)    perform  analyses   of   systems  in   their   operational 
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environments,  (2)  establish  quantitative  instructional 
standards,  (3)  provide  an  index  of  achievement  for  each 
behavioral  objective,  and  (4)  evaluate  alternative 
instructional  content,  approaches,  and  training  devices. 

In  particular,  instructional  system  development 
requires  that  performance  standards  are  identified  so  that 
the  most  efficient  approach  is  used  to  train  the  needed 
skills  and  knowledge  to  the  desired  level  of  performance. 
Such  performance  standards  imply  performance  measurement  for 
both  the  determination  of  desirable  approaches  to  training 
and  fcr  testing  student  performance. 

All  these  events  mentioned  above  should  be 
considered  positive  and  therefore  this  model  for 
instructional  system  development  could  also  be  used  and 
adapted  by  HAF. 

2.  A   Measurement    System   for     the    Operational 
Environment . 

It  will  be  necessary  to  develop  measurement  tools 
that  would  be  usable  in  the  operational  environment  of  HAF 
under  the  constraints  that  such  an  environment  implies. 
Within  the  context  of  this  requirement,  it  is  essential  that 
an  attempt  be  made  to  establish  a  list  of  parameters  to  be 
sensed,  and  the  pcint-of- view  taken  that  the  parameters 
should  be  derived  from  that  information  that  the  operational 
training  personnel  consider  to  be  meaningful  and 
significant. 

3.  An  Automated  Measurement  System. 

An  ancillary  objective  is  to  develop  a  measurement 
system  that  will  relieve  the  instructor  pilot,  to  a  maximum 
extent,  from  the  requirement  of  having  to  record  a  great 
deal  of  information  manually  on  the  basis  that  such  activity 


27 


degrades  his  ability  to  competently  instruct  his  student. 
This  does  not  imply  that  such  a  measurement  system  is  an 
attempt    at      automated  evaluation.  The    measurement      system 

should  certainly  include  means  for  transforming  and 
analyzing  performance  information,  but  ultimately  evaluation 
and    decision   for   training   control   is    a   human   function. 

Thus,  HAF  would  improve  and  optimize  the  guality  of 
training  of  its  combat  pilots  by  adapting  an  instructional 
system  development,  a  measurement  system  for  the  operational 
environment   and    finally   an   automated    measurement    system. 


B.       MEASUREMENT      BASEE   ON      COMBAT-CREW    TRAINING       INFORMATION 
NEEDS 

The  strategy  employed  by  the  OSAF  was  to  design  a 
measurement  system  that  could  acguire  that  data  identified 
as  meaningful  by  training  management  and  instructor 
personnel.  Data  were  gathered  from  sites  across  a  broad 
spectrum   of  combat-crew    training    programs   of    USAF. 

The  combat-crew  training  sites  from  which  data  were 
taken  are  listed  in  Table  2.  The  aircraft  sample  included 
heavy  (inter-  and  intra- theatre  cargo/transport,  and 
bomber)  and  high-performance  aircraft  (one-  and  twc-man 
interceptor  and  tactical  fighter) .  An  attempt  was  made  to 
(1)  consider  measurement  in  the  context  of  combat-crew 
training,  (2)  assess  measurement  already  included  as  well  as 
identify  potential  measurement  indicated  by  combat-crew 
training  personnel,  and  (3)  assess  the  constraints  placed  by 
the   environment    on    feasible,    usable    measurement   systems. 

From  the  six  types  of  aircraft  listed  in  Table  2  the  HAF 
pilots  fly  the  three  cf  them,  that  is  F-UE,  C-130E  and  A-7D. 
Thus,  the  DSAF's  attempt  and  data  found  are  of  great 
interest  for  HAF,  in  order  to  improve  training  performance 
information   for    its   ccmbat-crews. 
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TABLE    2 
Data   Collection   Sites 

Place  Aircraft 

Castle    AFB  B-52    F,    G    5    H 

AltUS    AFB  C-141A 

Dyess   AFB  C-130E 

Davis-Monthan    AFB  F-4    C,    D   S    E 

Tyndall   AFB  F-106    A   &    B 

Luke    AFB  A-7D 

*G€orge    AFB  F-4E 

*Ncrton    AFB  C-141A 

**Nellis   AFB  F-4E 


*    Special  emphasis   on   crew-performance    measurement 
**Special  emphasis  on   air-combat   maneuvers 
performance    measurement. 


C.       COMBAT-CREW    PERFORMANCE    MEASUREMENT 

For  the  six  aircraft  mentioned  in  Table  2,  a  ccmmon 
basis  for  measurement  was  established.  This  way  it  could  be 
possible  to  apply  a  mere  or  less  modular  approach.  Allowance 
must  be  made  for  special  aircraft  characteristics;  for 
example,  the  F-106  has  no  flaps  and  the  B-52  has  a  quite 
complicated  flap  retraction  routine  compared  to  other 
aircraft.  Measurement  was  treated  for  each  of  the  following 
maneuvers: 

(1)  Takeoff   and   Climb 

(2)  Pattern,    Land  and   Go-Around 

(3)  Instruments 

(4)  Formation 

(5)  Air-Air   Intercept 
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(6)  Air   Combat    Maneuvers 

(7)  Air   Refueling 

(8)  Ground   Attack 

(9)  Air   Drop   and  Air   Drop   Formation 

(10)  Radar    Navigation   and   Bombing. 

Prototype  measurement.  For  each  of  above  maneuvers,  the 
data  collected  from  the  sites  of  Table  2  were  compiled  into 
the  summary  form  shown  in  Figure  4. 1  Since  the  blanks  in  the 
summary  form  of  Figure  4. 1  indicate  needed  information,  and 
consequently  items  fcr  measurement  development,  these  forms 
were  termed  Prototype  Measurement  since  they  form  a  model 
after   which      measurement   could      be   patterned.  Details   and 

explanations  of  a  prototype  measurement  is  presented  in 
Appendix   A. 

Measurement  Specifications.  The  parameters  which  must 
be  sensed  to  permit  measurement  are  not  immediately  evident 
from  the  information  requirements  (Prototype  Measurement) , 
since  the  measure  specifies  the  output  of  a  computation,  and 
the  ccmputation  itself  must  be  known  before  the  inputs  to 
the   computation    (the    parameters)    can   be   determined. 

For  example,  Figure  4. 1  indicates  that  a  measure  of 
centerline  deviations  is  desired  during  the  takeoff  roll.  It 
is  clear  that  the  distance  between  the  aicraft  position  on 
the  runway  and  the  runway  centerline  is  a  parameter  needed 
for  measurement.  The  desired  measurement  might  be  simply  the 
average  difference,  or,  conceivably,  might  involve  the  rela- 
tionship between  centerline  deviation  and  heading  (or 
lateral-G,  or  brake  application)  and  thereby  indicate  the 
need  for  other  parameters  to  execute  the  calculations  for 
measurement.  Further,  the  measurement  calculations  must  be 
made  (as  indicated  in  Figure  4.1  )  from  the  application  of 
takeoff   power   until      rotation,      implying   the  need      for    ether 
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TAKEOFF    S    CLIMB* 

CONDITIONS: 

73ross   wt:_  Wing:_   _/  Runway: / 

Temp.: '  "Alt. Set.: Field  Elev. Form  Pos.  : 

TAKEOFF    ROLL:     (To    power  until   rotation) 

Power   SeT:~  TenterTIne   t)ev.  :    Min,    Max,    Av. 

Reject   Speecl:    Computed  Heading: Min,    Max,    Av. 

Time: Dist: Bank:    fi   Max,    L    Max 

ROTATION:     (Nose    gear   off  until   pitch    att.    established) 

F of. "Speed: STaTJ.    Trim: 

Pitch:      Rate: Bank:_ 

Final:"  Centerline   Dev. : 

Overshoot: Heading: 

LIFTOFF:     (Pos.    Vert.    Vel. ) 
"Xirspeed:                             "Pitch:  Bank:  Hdg: 

Vert.    Vel7~"A"fter : Sec: 

GEAR-UP:     (Handle   _up   until   £ear-up_  &   locked) 

G*ear-u*p  SpeeaT_  _  V.v.    iniTT: V.  7.    Final: 

Pitch: Bank: Hdg: 

FLAES    DP:     (Start    up   to    full   up)     ** 

TrimT_                                                W=52   Only    IAS  PITCH    ALT  VV  TRIM 

Pitch:        Bank:         Hdg:             Start             x  x          x  x  x 

A/S    (IFTT)           TFINAL)               1st    Pos        x  x           x  x  x 

VV    (INIT1               (FINAL)               2nd    Pos         x  x           x  x  x 

ALT     {INIT}"2_       (FINAL)  ~         Full                 x  X           X  x  x 

CLIMB   S    LEVEL    OFF:    (Defends    on   Flight   Plan) 

INIT    FINAL 

£WR    A^S  MACH    HDG    ALT    ALT    PITCH    TRIM 

Accelerate                             xx  xxxx           x             x 
Climb   A/S       (#1) 

(#2)                  xx  xxxx           x             x 

Climb   MACH                              xx  xxxx           x             x 
Level-off    fAlt-10*  VV) 

(to   Cruise)    xx  xxxx           x             x 


*  Also,  mandatory  communication  &  instances  where  A/C 

limits  are  exceeded. 
**F-106  has  no  flaps 


Figure  4. 1    Example  of  Prototype  Measurement. 
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parameters  to  indicate  when   the  measurement  interval  starts 
and  stops. 

Since  the  pilots  of  HAF  fly  at  most  the  same  aircraft1 
as  USAF  pilots,  the  data  gathered  from  these  USAF  bases  are 
of  great  interest  for  HAF.  Of  course,  of  greatest  interest 
are  the  data  of  the  three  similar  aircraft  (F-4E,  A-7D,  and 
C-130E),  but  the  data  related  to  the  other  aircraft  should 
also  be  of  great  interest.  These  may  be  applied  to  ether 
aircraft  that  pilots  of  HAF  fly  (e.g.  F-1C  Mirage,  or 
F-5A/B) ,  or  aircraft  that  HAF  will  be  procured  in  the  near 
future. 


iThe  aircraft  that  the  HAF's  pilots  fly  and  are  similar 
to  the  USAF  aircraft  bases  from  where  data  were  gathered 
are:  F-4E,  A-7D,  and  C-130E. 
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V.  DEBIVATION  OF  CCMBAT-CREW  PERFORMANCE  MEASUREMENT. 

One  cf  the  principal  products  of  the  USAF  study  was  the 
definition  of  performance  measures  appropriate  to  combat- 
crew  training  needs.  That  study  was  based  on  interviews  with 
instructors  and  training  management  and  measurement  defini- 
tions were  evolved  in  the  following  steps  ; 

(1)  The  varied  requirements  posed  by  the  six  aircraft 
and  missions  were  consolidated  into  a  common  framework  which 
permitted  isolation  of  measurement  modules  (measurement 
commonality)  ; 

(2)  Discussions  of  operational  training  information 
needs  were  formalized  to  indicate  in  a  checklist  fashion  the 
measurement  development  needed  (prototype  measurement)  ; 

(3)  Measurement  parameters  and  (4)  measurement  specifi- 
cations were  produced  together,  but  are  presented  separately 
for  hardware  and  software  implications,  respectively  ; 

(5)  Measurement  and  analysis  for  crew  communications 
recording  were  examined  to  provide  means  of  examining  crew 
interaction  and  individual  contributions  to  total  man- 
machine  system  performance. 

A.   MEASUREMENT  COMMONALITY 

Common  measurement  was  executed  to  permit  the  design  of 
one  simple  and  practical  measurement  system,  and  eliminate 
the  need  for  a  totally  unique  measurement  system  for  each 
aircraft.   This  measurement  system  could  apply  in  HAF. 
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1.   Common  Training  Phases. 

In  the  USAF  program  the  first  step  was  to  compare 
similar  training  phases  for  each  aircraft,  in  order  to  esti- 
mate the  commonality  measurement  requirements,  as  it  is 
shown  in  Table  3.  In  cases  where  X-Y2  data  are  required  for 
measurement  it  was  found  that  these  data  could  be  obtained 
by  using  expensive  equipment  (such  as  a  multiple-target 
tracking  radar)  or  with  dif f icult-to- process  recordings, 
while  equivalent  results  could  be  obtained  with  video/photo 
sensors.  As  we  can  see  in  Table  3,  almost  all  cases  of  X-Y 
data  requirements  can  be  met  by  using  video/photo  recording. 
Those  cases  where  the  X-Y  data  cannot  be  obtained  with 
video/photo  sensors  are:  (1)  lateral  drift  across  the 
runway  during  transition,  (2)  relative  position  of  aircraft 
during  intercept  prior  to  lockon,  (3)  enroute  cross-track 
error  during  airdrop,  (4)  inflight  ranging  (out  of  sight) 
during  formation,  and  (5)  space  paths  of  multiple  aircraft 
during  air  combat  maneuvers. 

Although,  phcto-sensors  are  widely  used  by  the  HAF, 
video  is  not.  So,  the  use  of  video  films  would  be  recom- 
mended because  video  is  a  modern  technology  equipment  with 
many  advantages  (e.g.,  film  does  not  need  development  and 
therefore  can  be  used  immediately  after  flying  during 
debriefing) . 

Not  all  maneuvers  were  taught  at  all  sites  from 
which  data  was  taken.  For  example,  the  operational  C-130 
squadron  did  not  explicity  train  transition  maneuvers; 
nevertheless,  competent  information  was  obtained  for 
measurement.  Since  all  combat  maneuvers  were  not  taught  at 
the  combat-crew  training   squadrons,   they  had  gaps   in  data 


2The  X-Y  data  are  not  in  the  form  of  a  tabulation  of  X- 
and  Y7values,  but  they  are  position  information  such  as  the 
relationship  between  tanker  and  refueling  aircraft  from  the 
tanker  lights. 
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TABLE  3 
Requirements  For  X-I  Data 

Ehase  Parameter  Obtainable 

wi!| 
Vicieo/Photo? 

1  Transition  Ground  Track  RA* 

2  Centerline  Dev  RA 

3  Lat.  Drift  No 

4  Threshold  RA 

5  Dist.  Down  RNWY  RA 

6  Spacing  RA 

7  Intercept  TGT.  Azimuth  No 

8  (Prior  To  Lockon)  TGT.  Elevation  No 

9  TGT.  Range  No 

10  TGT.  Range  Rate  No 

11  TGT.  Aspect  Angle  No 

12  Air  Refueling  Tanker  Range  RA 

13  Tanker  Range  Rate  RA 

14  Centerline  Displ.  Yes 

15  Lights  Up  Yes** 

16  Lights  Down  Yes** 

17  Lights  Fore  Yes** 

18  Lights  Aft  Yes** 

19  Altitude  Error  RA 

20  Air  Drop  Cross  Track  Error  No 

21  Position  Error  Yes 

22  Range  from  Lead  RA 

23  Bearing  from  Lead  RA 

24  A  Altitude  from  Lead    RA 

25  Actual  Air  Release  Pt.  Yes 

26  Formation  Range  RA 

27  Range  Rate  RA 

28  .  Bearing  RA 

29  Ground  Attack  TGT.  Slant  Range  RA 

30  Aim  Point  Error  Yes 

31  Bomb  Fall  Line  Yes 

32  Flight  Path  RA 

33  Spacing  RA 

34  Dart  Firing  Range  RA 

35  Azimouth  RA 

36  Elevation  Yes 

37  Air  Combat  TGT.  Range  RA 

38  TGT.  Range  Rate  RA 

39  TGT.  Aspect  Angle  RA 

40  TGT.  HDG  Cross  Angle    No 

41  Elevation  RA 

42  Space  Path  No 


*  RA=Reduced  Accuracy 

**Cbtainable  with  Video/Photo  System, 
tut  not  easily  otherwise. 
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collection.    They    tried   to   fill    these    gaps   by   cross-checking, 
where   possible,    with    ether   aircraft    training. 

The  following  are  observations  of  the  degree  of 
commonality  of  training  phases  for  the  six  USAF  aircraft  and 
HAF's    aircraft    also: 

(1)  TRANSITION  (TE)  .  Transition  is  a  phase  of 
combat-crew  training.  Transition  maneuvers  appear  in  a 
common  fashion,  but  the  manner  in  which  they  are  performed 
is  significantly  different  for  different  aircraft.  This 
phase    has    leen    already  adapted   by  HAF. 

(2)  INSTRUMENTS  (INST)  .  Instruments  is  a  common 
flight  phase  for  all  aircraft  (also  adapted  by  HAF)  and 
consequently  common  measurement  (criteria)  is  conceptually 
possible. 

(3)  FORMATION  (FORM) .  Formation  flight  is  performed 
by  all  six  aircraft  as  a  means  to  optimally  employ  composite 
flight  and  provide  individual-ship  effectiveness.  It  is 
considered  as  a  common  flight  phase  among  all  aircraft, 
however,  a  number  of  types  of  formation  exist  for 
specialized  missions,  each  appropriate  only  to  specific 
aircraft.  The  formation  phase  is  widely  used  in  HAF  by  high 
performance  aircraft  (as  in  F-4E,  A-7D,  F-1C,  F-5A/B)  and  is 
very  limited  among  multi-engine  aircraft  (C-130,  DC-6, 
Albatros) .  Measurement  differences  will  occur  between  the 
categories  of  high  performance  aircraft  and  multi-engine 
heavy   aircraft. 

(4)  AIR-AIR  INTERCEPT  (AA) .  Air-to-air  intercept 
and  weapons  delivery  utilizing  airborne  radar  is  accom- 
plished with  only  the  F-U  and  F-106  aircraft  (in  HAF  with 
F-4E  and  F-1C).  While  the  F-4,  F-1C  and  F-106  maneuvers  and 
equipment  differ,  the  same  basic  measurement  requirements 
are  presented.  Almost  the  same  basic  measurement  require- 
ments could  also  apply  in  F-5A/B  of  HAF,  against  the  event 
that    air-air   intercept   is   accomplished   by    ground    radar. 
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(5)  BASIC  FLIGHT  MANEUVERS  (BFM)  AND  AIR  CCMEAT 
MANEUVEES  (ACM) .  BFM  and  ACM  are  grouped  together  in  F-4 
training,  and  could  be  grouped  also  in  F-5A/3  and  F- 1C  of 
HAF,  while  BFM  and  Formation  are  grouped  together  in  A-7 
training.  The  BFM/ACM  grouping  was  maintained  for  purposes 
of  measurement  problem. 

(6)  AIR  REFUELING  (AE)  .  Air  refueling  can  cccur 
with  fcur  of  the  six  USAF  aircraft  (B-52,  F-106,  F-4,  A-7), 
but  is  only  considered  a  difficult  maneuver  for  the  E-52. 
In  HAF  air  refueling  can  occur  only  in  F-4E  and  A-7  but  is 
not  used  since  there  are  not  any  tankers  available. 

(7)  GROUND  ATTACK  (GA)  AND  DROP.  A  number  of 
training  phases  are  devoted  to  F-4  and  A-7  (both  aircraft 
are  available  by  HAF)  ground  attack  (Day,  Tactical,  Night, 
with  various  weapons  and  delivery  modes) ,  but  common 
measurement  was  judged  to  be  appropriate. 

Air  Drop  (ccmbat  airlift  mission)  maneuvers  for 
C-141  and  C-130  training  perhaps  superficially  resemble 
level-flight  ground  attack,  but  quite  different  measurements 
are  posed.  Ground  attack  and  drop  phases  have  been  also 
adapted  by  HAF  but  in  this  casethe  measurement  is  different. 

(8)  RADAR  NAVIGATION  AND  BOMBING  (RNB) .  Navigation 
by  use  of  radar,  and  subsequent  delivery  on  target  of  either 
ordnance  or  cargo,  occurs  with  most  of  the  aircraft  of  the 
sample  except  the  F-106,  and  compatible  mission  performance 
measurement  is  indicated.  In  HAF  this  phase  is  used  by  all 
aircraft  except  these  that  are  not  equipped  with  radar 
(e.g.  ,  F-5A/B)  . 

2-   Summary  of  Training  Phases. 

The  examination  of  the  USAF  study  shows  that 
measurement  (criteria) ,  for  the  training  phases  of  the 
sample  of  six  USAF  aircraft,  are  composed  as  follows  : 

Transition 

Instruments 
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Formation 

Air-to-air   Intercept 

Air   Combat 

Air  Refueling 

Ground  Attack 

Air  Drop 

Radar  Navigation  and  Bombing 
All  of  these  training  phases  (except  air  refueling) 
are  widely  used  by  the  aircraft  of  HAF.  From  both  studies, 
DSAF's  and  present,  it  is  concluded  that  not  all  of  the 
training  phases  are  applicable  to  all  types  of  aircraft.  For 
this  reason,  it  is  required  to  ignore  the  measurement 
developed  in  cases  where  a  specific  phase  could  not  be 
applied. 

3 .   Common  Measurement  for  Maneuvers 

Each  phase  of  flight,  tentatively  considered  to 
permit  common  measurement, was  examined  for  detailed  measure- 
ment requirements.  All  needed  mesurements,  for  each  maneuver 
used  in  the  USAF  study,  were  provided  by  (1)  interview  notes 
with  Instructors  and  Training  Managers,  (2)  Aircraft 
Techical  Orders,  Dash-one  flight  manuals  for  each  aircraft, 
(3)  Phase  Manuals,  (4)  Instructor  Guides,  and  (5)  other 
specialized  documents.  Important  information,  as  judging 
factors  and  common  errors  were  noted  for  each  maneuver, 
along  with  other  important  remarks  that  an  instructor  pilot, 
or  training  manager,  would  consider  and  thus  translated  into 
objective  measurement. 

For  example,  measurement  requirements  were  noted  for 
each  aircraft  during  takeoff  and  climb-out  maneuvers  (Runway 
Roll,  Rotation,  Liftoff,  Gear-up,  Flaps-up,  Climb  and 
Level-off).  A  matrix  of  measurement  requirements  was  thus 
produced,  allowing  comparison  across  aircraft.  It  was  noted 
that  takeoff  and  climb-out  are   essentially  the  same  for  all 
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aircraft,  with  the  exception  of  the  importance  of  some 
maneuvers  (e.g.,  rotation  is  critical  with  the  F-106),  vari- 
ations in  aircraft  design  (e.g.,  F-106  has  no  flaps,  B-52 
involves  a  complicated  flap  schedule) ,  and  climb  profiles 
depend  on  the  specific  mission  and  clearance. 

Although  there  were  some  differences,  in  measurement 
requirements  among  different  types  of  aircraft,  the 
similarities  proved  to  be  a  lot  more.  Also,  a  modular 
approach  to  measurement  was  suggested,  that  is,  measurement 
could  be  produced  by  examining  every  maneuver  in  sequence 
for  every  type  of  aircraft. 

All  of  the  above  mentioned  maneuvers  are  used  by  the 
Hellenic  aircraft  and  thus  common  measurement  for  maneuvers 
in  a  modular  approach  could  also  be  considered  by  all 
aircraft. 

B.   PEOTCTIPE  MEASUREHENT. 

After  the  examination  of  measurement  commonality  of 
training  phases  and  maneuvers,  the  USAF  study  developed,  in 
the  form  of  formatted  measurement  outputs,  examples  of  the 
required  information  for  training.  That  is,  if  a  measure  of 
centerline  deviation  was  indicated  to  be  desirable,  this 
would  be  noted.  In  this  way  they  recorded  all  of  the  known 
information  requirements  for  a  given  phase  of  flight,  and 
then,  they  formed  the  data  found  into  a  format  to  resemble 
measurement  output.  This  output  is  termed  here  as  Prototype 
Measurement. 

An  example  of  prototype  measurement  is  displayed  in 
Figure  4.1  for  takeoff  and  climb.  Details  and  explanations 
of  this  (prototype  measurement)  example  is  presented  in 
Appendix  A.  A  blank  or  x  in  Figure  4.1  indicates  one  or 
more  numerical  entries  to  be  determined  as  a  result  of 
measurement.  For  every  one  of  the  training  phases  similar 
measurement  was  developed. 
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In  the  USAF  study  the  prototype  measurement  was  produced 
for  the  following  trainig  phases: 

(1)  TRANSITION.  For  convenience,  transition  was  divided 
into  (a)  Takeoff  and  Climb,  and  (b)  Pattern,  Land  or 
Go-arcund.  As  previously  noted,  measurement  appropriate  to 
the  composite  of  six  aircraft  is  indicated;  the  F-106  and 
B-52  require  special  treatment. 

(2)  INSTRUMENTS.  Measurement  for  instrument  flight  is 
treated  as  the  sum  of  (a)  basic  aircraft  control 
performance,  and  (b)  navigation  performance  with  respect  to 
air  traffic  control  requirements.  While  required  measure- 
ment modules  for  each  of  these  classes  of  measures  can  be 
specified,  it  is  difficult  to  present  detailed  measurement 
except  for  specific  clearances  and  published  procedures. 

(3)  FORMATION.  Measurement  for  formation  flight  is 
super-imposed  onto  mission  performance  measurement,  thus 
making  formation  measurement  difficult  to  isolate.  In 
particular,  tactical  formation  performance  was  not  clearly 
identified  in  this  study  in  an  objective  quantitative 
fashion.  It  may  be  necessary  to  rely  heavily  on  instructor 
subjective  measurement  (which  may  be  quite  satisfactory  when 
the  instructor  is  in  a  position  to  observe  performance  ) , 
except  for  measurement  associated  with  join-up,  close  forma- 
tion, and  in-trail  formation. 

(4)  INTERCEPT.  In  order  to  be  specific,  intercept 
measurement  was  based  primarily  on  the  F-106,  however,  the 
intercept  problem  is  essentially  the  same  as  the  F-4.  Of 
cource,  a  two-man  crew  performance  is  expected  to  be  better 
in  the  F-4.  There  is  a  radar  observer  to  perform  the  scope 
work  and  differences  in  the  equipment  and  capability  suggest 
that  slightly  different  strategies  might  be  employed. 
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(5)  AIF.  COMBAT  MANEUVERS.  Prototype  air  combat  maneu- 
vering measurement  dealt  primarily  with  set-ups  during 
initial  training  and  for  dart3  firing. 

(a)  Air  combat  set-ups  involve  placing  attacking  and 
defending  aircraft  in  fixed  initial  positions,  then  freeing 
them  to  perform  a  maneuver,  and  subsequently  judging  from 
the  final  position  whether  the  maneuver  was  properly 
performed  and  whether  proper  advantage  of  the  tactical  situ- 
ation was  taken.  Thus,  measurement  can  be  directed  to 
description  of  the  maneuvers  performed  (e.g.,  hard  turn, 
hi-lo-speed  yo-yo,  scissors,  barrel  roll,  etc.),  and  to 
determining  whether  a  given  student  was  able  to  improve  his 
situation. 

(b)  For  dart  firing  the  prototype  measurement  assumes  a 
butterfly  pattern  or  the  equivalent.  A  pass  is  made  over  the 
target,  a  time  hack  is  taken  crossing  the  dart,  the  pilot 
must  circle  back  to  make  an  intercept  to  put  a  hole  within 
the  target  in  a  given  amount  of  time.  Thus,  the  time  and 
hits  on  each  pass  is  measured;  additionally  the  range, 
azimuth,  and  elevation  at  the  beginning  and  end  of  firing 
describe  the  firing  position.  Fouls  are  called  for  low 
airspeed  and  for  firing  within  a  minimum  firing  range. 

(6)  AIR  REFUELING.  As  Air  refueling  is  especially  diffi- 
cult in  E-52  combat-crew  training,  the  prototype  measurement 
was  tailored  to  the  E-52  tasks  and  to  Strategic  Air  Command 
requirements. 

(7)  GROUND  ATTACK.  During  training,  ground  attack  is 
divided  into  ground  attack,  ground  attack  night,  ground 
attack  tactical.  Ground  attack  measurement  was  dictated  by 
information  needed  for  standard  error  analysis  of  weapons 
delivery  accuracy  and  by  ground  attack  procedures.    Some  of 


3The  dart  is  a  kind  of  air  to  air  target,  towed  by  a  tow 
aircraft.  3  J 
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the  measures   are  specially   designed  to   apply  to   the  A-7D 
heads-up  display. 

(8)  AIR  DROP.  Extensive  prototype  measurement  was 
dictated  for  the  Comtat  Airlift  Mission  since  very  detailed 
procedures  are  adhered  to  throughout  the  many  portions  of 
the  mission.  Crew  performance  is  especially  important  in 
this  mission. 

(9)  RAEAR  NAVIGATION  AND  BOMBING.  Prototype  measurement 
for  Radar  Navigation  and  Bombing  is  heavily  dependent  on  the 
characteristics  of  the  avionics  used.  As  only  the  B-52  is 
equipped  with  low-level  terrain  avoidance  radar  (of  the 
aircraft  sampled  in  this  study),  measurement  for  these 
maneuvers  has  been  tailored  to  this  application. 

Of  course,  from  the  examination  of  the  prototype 
measurement  we  can  see  that  the  measurement  requirements  are 
very  extensive  and  complex.  For  example,  it  may  be  seen  that 
to  describe  just  Takeoff  and  Climb,  it  is  necessary  to 
compute  50-100  numbers.  Subsequently,  if  full  mission 
measurement  is  required,  including  transition,  instruments, 
formation,  and  weapons  delivery,  it  will  require  a  very 
large  set  of  descriptive  numbers.  But  probably,  this  detail 
is  very  necessary  to  support  the  training  process.  The 
instructor  may  need  considerable  detail  to  perform  his  job 
well.  As  a  consequence,  the  prototype  measurement  could  be 
very  useful  for  HAF,  since  today  the  evaluation  of  aircrew 
proficiency  in  skills  associated  with  advanced  flying 
training  is  primarily  based  upon  the  subjective  judgement  of 
instructor  pilots. 

C.   MEASUREMENT  PARAMETERS. 

Measurement  is  the  process  of  producing  measures  which 
are  indices  of  performance  such  as  the  conditions  existing 
at   the  time   of   weapon   release   during   ground   attack, 
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deviations  from  the  clearance  during  IFR  flight,  and  flight 
conditions  at  lift  off.  The  measures  are  commonly  computed 
from  flight  variables  (e.g.,  altitude,  airspeed,  heading, 
etc.)  and  other  raw  information  which  must  be  recorded  at 
some  time;  these  are  termed  measurement  parameters.  The 
measurement  parameters  must  be  specified  so  that  the 
required  sensors  and  recording  equipment  can  be  determined. 
However,  since  the  measures  are  the  result  of  a  computation, 
the  details  of  the  computation  must  be  known  so  that  the 
inputs  to  the  computation  (the  parameters)  car  be 
established. 

Figure  5.1  depicts  the  relationship  between  the  speci- 
fied measures,  the  computation,  and  the  measurement  parame- 
ters. The  corresponding  data  processing  is  shown  in  Figure 
5.2.  It  is  assumed  that  the  flight  maneuvers  will  be 
divided  into  segments,  so  that  different  measures  may  be 
computed  as  appropriate  for  each  segment  (for  example, 
different  measures  are  required  during  takeoff  roll  than 
during  climb-  out) .  Consequently,  it  may  be  seen  that  the 
method  of  determining  when  to  start  and  stop  the  computation 
of  a  specific  measure  may  require  the  recording  of  measure- 
ment parameters  in  addition  to  those  required  during  measure 
computation  (e.g.,  the  recording  of  weight-of f-the- wheels  to 
indicate  liftoff). 
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PARAMETER 
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Figure  5.2     Example   Raw  Data   Processing 
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In  Figure  5.1,  the  output  measures  (0)  correspond  to  the 
information  requirements  symbolized  by  the  blanks  in  the 
prototype  measurement  forms;  that  is,  if  the  prototype 
measurement  forms  indicated  that  a  measure  of  centerline 
deviation  is  needed,  then  parameters  must  be  recorded  and 
computations  developed  which  will  answer  the  required 
measure,  the  following  types  of  parameters  may  be  needed  in 
addition  to  the  basic  test  parameters  (M)  (for  example  just 
mentioned,  the  basic  test  parameter  would  be  a  recording  of 
the  deviation  from  the  runway  centerline  during  takeoff 
roll):  (1)  parameters  for  implementing  logic  to  start  and 
stop  measurement  computations  (S) ,  (2)  information  related 
to  desired  performance  (D) ,and  (3)  error  information  derived 
from  the  differences  between  actual  and  desired  performance 
(E) .  In  short,  given  output  measures  (0) ,  to  determine 
other  parameters  which  must  be  sensed  (M,S,D,E),  it  is 
necessary  to  determine  logic  and  computations  to  be  used  in 
measurement  data  processing  (i.e.,  the  measurement 
algorithms)  . 

Assuming  automated  measurement,  i.e.,  parameters  are 
automatically  recorded  for  subsequent  computer  analysis,  the 
primary  details  of  measurement  computation  (Figure  5.2  shows 
a  representative  flow  diagram)  are  presented  in  Figure  5.3 
for  each  maneuver  and  maneuver  segment  of  combat-crew 
training  phases.  The  figure  5.3  indicates  the  name  of  each 
measure,  the  specific  function  to  be  computed,  and  the 
start/stop  conditions  for  controlling  computation.  For 
example,  centerline  deviation  during  the  takeoff  roll  is 
desired  output  information,  the  average,  minimum  and  maximum 
deviation  are  the  specific  computations  which  should  be 
performed  between  brake  release  and  rotation.  Comments  are 
also  provided  as  considered  appropriate  by  the  analyst  to 
point  up  alternatives,  or  where  problems  may  be  encountered 
during  design. 
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Since  figure  5-3  indicates  the  functions  to  be  computed, 
the  conditions  under  which  computation  should  occur,  and 
indirectly  the  source  information  upon  which  to  base  compu- 
tation, the  basic  information  is  provided  to  allow  prepara- 
tion of  computer  programs  for  automatic  measurement. 

When  a  list  of  parameters  was  developed  to  show  all  the 
required  sensors  and  recording  needed  for  total  measurement, 
it  was  seen  that  the  resulting  parameters  could  be  placed  in 
the  following  overlapping  categories:  (1)  pictorial  infor- 
mation (e.g.,  out-  tie-window  radar),  (2)  analog  information 
(e.g.,  time-varying  quantitative,  such  as  airspeed), 
(3)  discrete  information  (e.g.,  weapon  release),  (4)  audio 
information  (e.g.,  communications),  and  (5)  desired 
performance  and  existing  conditions.  These  parameters  were 
later  assigned  to  alternative  devices  for  data  acquisition. 
After  tradeoff  analyses  were  conducted,  a  hybrid  audio- 
video/photo  -digital  recording  system  was  adopted.  Although 
many  parameters  could  be  acquired  by  either  video-photo  or 
digital  recording  devices  {to  allow  a  partial  system  to  have 
a  stand-alone  capability) ,  tentative  parameter  allocations 
are  listed  in  Tables  4  and  5.  Audio  recording  will  be 
accomplished  with  either  video-photo  or  digital  recording 
devices.  Desired  performance  and  conditions  are  manually 
derived  from  briefing/debriefing  sessions  and  documents. 
Additionally,  spatial  coordinates  (X-Y  data)  may  be  obtained 
if  data  are  collected  on  an  instrumented  range  including 
tracXing-radar  equipment,  although  equivalent  information 
may  be  available  from  video-photo  recording  as  shown  in 
Table  3. 

After  detailed  trade-off  analyses,  the  use  of  video  or 
photo  techniques  was  emphasized  due  to  lower  costs,  flexi- 
bility of  application  and  simpler  development  compared  to 
other  all-electronic  techniques;  however,  the  problems  asso- 
ciated with  cockpit  installation  and  an  unfortunate  tendency 
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TABLE    4 
Parameters   for   Digital   Data   Acquisition 
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for  loss  of  data  should  be  noted,  but  it  is  believed  neces- 
sary to  accept  these  deficiencies  for  low-cost  combat-crew 
training  application. 

Accuracy  requirements  for  measurement  parameter  sensing 
and  recording  are  listed  at  the  right-hand  side  of  Table  4. 
These  accuracies  are  referenced  to  the  information  displayed 
to  the  crew  (for  example,  the  required  airspeed  tolerance  is 
plus  or  minus  1%  of  the  deviation  between  recorded  values 
and  those  displayed  to  the  pilot  on  his  airspeed  indicator) 
since  the  criterion  given  the  crew  is  that  they  maintain 
vehicle  parameters  within  specified  tolerances  referenced  to 
their  instruments  (typically  5-10  knots  for  airspeed 
control) .  The  tolerances  listed  in  Table  4  are  approxi- 
mately 1/10th  the  tolerances  required  of  the  crew. 

As  a  result,  (a)  if  HAF  has  to  make  a  choice  between 
video/photo  and  digital  recording  approaches  to  measurement 
system  design,  then  the  video/photo  recording  would  be 
chosen  for  cost,  information  provided,  flexibility  and  ease 
of  use.  (b)  A  hybrid  system  combining  the  advantages  of 
both  is  preferable  to  either  type  of  recording  alone.  Due 
primarily  to  cost,  the  bulk  of  measurement  parameters  would 
be  derived  from  a  video/photo  system,  and  the  remainder  with 
a  small  digital  recording  capability. 

D.   MEASUREMENT  DESCBIPTIONS 

The  gross  operations  involved  in  measurement  computation 
are  presented  in  the  flow  diagram  in  Figure  5.2  Each 
parameter  must  be  sampled  {at  a  sampling  rate  of  2,  10  or  20 
times  a  second,  depending  on  the  application)  and  tested  to 
determine  if  conditions  are  appropriate  to  start  measurement 
computation,  and  later  to  stop  computaton.  During  the 
measurement  interval  (or  at  specific  conditions,  e. g. , 
flaps-up),   one   of  the  following  statistics  is  calculated: 
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TABLE  5 
Parameters  for  Video/Photo  Data  Acquisition 

PHASE  PARAMETERS 


Transition     Runway  Centerline  Deviation,  Lateral 

Drift,  Threshold  Crossing,  Distance  Down 
Runway,  Ground  Track. 

Instruments     TACAN  (Fregu.,  Course  Set,  Course  Error, 

Bearing,  DME) • 

VOR  {Fregu./  course  Set,  Course  Error, 
Bearing)  ; 

ILS  (Fregu.,  Localizer  Error,  Glide  Slope 
Error,  Marker  Beacon) . 


Intercept 


Target  use  of  ECM,  Maneuvering 
Radar:  Azimuth.  Elevation,  Range,  Range 
Rate,  Range  Gate,  Steering  Dot  Error, 
Firing  Circle  Radius,  Lockon  Pulse. 
IF  Gain,  Video  Gain,  Erase  Intensity. 


Refueling      Range  to  Tanker.  Range  Rate,  Probe 

Engagement,  Centerline  Displacement, 
Lights  (Dp,  Down,  Fore,  Aft) ,  Attitude 
Error. 

Air  Drop       Crcsstrack  Error,  Groundspeed,  Terrain 

Clearance,  Range/Bearing/ AAltitude  from 
Lead  A/C,  Red/Green  Drop  Lights,  Actual 
Air  Release  Point. 

Formation      Spacing:  Range,  Range  Rate,  Bearing, 

AAltitude. 

Ground  Attack   Target  Slant  Range,  Aim  Point  Error,  Eomb 

Fall  Line,  Flight  Path  Error,  Spacing 
in  Range  Pattern. 

Dart  Firing     Range,  Azimuth,  Elevation,  Hits. 

Air  Combat     Target  Range,  Range  Rate,  Aspect  Angle, 
Maneuvers      Heading  Crossing  Angle,  Elevation. 
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(1)  the  value  of  a  parameter,  (2)  minimum  value,  (3)  maximum 
value,  (4)  mean,  or  (5)  standard  deviation.  Thus,  measure- 
ment is  defined  by  specifying  start/stop  conditions  and  one 
of  the  five  statistics. 

An  example  of  a  measurement  specification  patterned  in 
this  fashion  is  shown  in  Figure  5.3.  Measurement  specifica- 
tions were  produced  for  each  common  training  phase. 

while  these  specifications  initially  assume  automatic 
recording  and  computation,  they  can  also  be  used  to  describe 
manual  data  processing  (e.g.,  a  discrete  signal  indicating 
wheels-up) .  These  parameters  can  also  be  used  to  start  and 
stop  manual  processing  such  as  scanning  for  out-of-tclerance 
conditions.  During  the  measurement  interval,  the  value, 
minimum  or  maximum  cf  a  parameter  may  be  determined  manu- 
ally, but  manual  processing  for  computation  of  a  mean  or 
standard  deviation  is  judged  to  be  excessively  laborious  and 
time-consuming  since  a  large  number  of  data  samples  (at  2  or 
more  times  a  second)  is  necessary.  Consequently,  the 
measurement  specifications,  as  exemplified  in  Figure  5.3, 
are  suitable  for  (1)  defining  software  for  digital  computer 
measurement  processing,  or  (2)  manual  processing  procedures. 

Today,  HAF  specifies  its  measurement  specifications 
mainly  by  manual  processing  procedures.  Lately,  HAF  has  been 
interested  in  computer  measurement  processing.  Thus,  HAF 
should  consider  the  measurement  specifications  mentioned  in 
this  section. 

E.   COMMUNICATION  ANALYSIS 

Measurement  for  crew  performance  overlays  the  system 
performance  measurement  thus  far  discussed.  The  performance 
demonstrated  in  A-7  or  F-106  aircraft  clearly  involves  only 
one  man;  however,  the  same  mission  may  be  flown  by  two  men 
in  the   F-4  aircraft,    requiring  additional   measurement  to 
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investigate  crew  interaction  and  diagnose  individual 
performance  contributions.  The  combat  airlift  mission  in 
the  C-130  and  C-141  aircraft  involves  such  close  coordina- 
tion of  pilots,  navigators,  loadmasters  and  engineers  that 
it  is  difficult  to  isolate  an  individual's  performance  (even 
the  crewmembers  themselves  cannot  be  sure  of  the  adeguacy  of 
their  performance) . 

An  individual's  performance  can  be  assessed  by  relating 
measurement  at  his  workstation  to  overall  system 
performance;  this  type  of  measurement  is  subsumed  under  the 
previous  measurement  discussion.    However,   the  interaction 

between  crewmembers   reguires  analysis   of  communications 

presenting  somewhat  different  requirements  than  system 
performance  measurement.  The  following  paragraphs  present 
important  topics  related  to  crew  performance  measurement. 

1 .   Communication  Measurement  Categories. 

Communications  measurement  must  treat  at  least  two 
gross  types  of  crew  interaction:  (1)  information  is 
exchanged  to  aid  another  crewmember  in  performing  his  duties 
(e.g.,  when  the  F-4  Weapon  System  Officer  acts  as  a  good 
"copilot") ,  and  (2)  a  crewmember  provides  a  directive  role 
in  guiding  another's  performance  (e.g.,  when  the  F-4  Weapon 
System  Officer  provides  directive  commentary  to  the  Aircraft 
Commander  in  air-air  intercept) .  In  the  later  case  direct 
links  between  auditory  commands  and  system  performance  can 
be  identified,  allowing  communication  to  be  measured  in 
terms  of  resulting  performance  changes. 

Six  categories  of  measurement  related  to  information 
transfer  were  examined: 

(1)  Timing.  Measures  of  information  timing  should 
relate  to  (a) jammimg  more  important  messages,  (b)  providing 
information  at  the   wrong  time,    (c)    delay  in   providing 
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information,  and  (d)  providing  information  at  a  rate  not 
permitting  effective  response  by  another  crewmember. 

(2)  Accuracy.  Measures  of  accuracy  reguire 
comparison  of  what  is  said  in  relation  to  the  measured  situ- 
ation (e.g.,  was  altitute  reported  correctly?). 

Q)  Brevity.  As  radio  and  interphone  traffic  often 
exceed  channel  capacity  in  combat,  measurement  should 
address  communication  duration  and  comparison  brevity  code. 

(4)  Number  and  Frequency,.  Also  in  relation  to 
communication  brevity,  the  number  of  communications  and 
frequency  cf  communications  can  be  measured. 

(5)  Information  content.  Measures  of  information 
transmitted  per  unit  time  are  quite  important  although 
usually  practically  difficult  to  obtain.  Other  measures  such 
as  time,  number  and  frequency  are  often  confounded  since  a 
crewmember  may  convey  much  information  in  a  short  time  or 
few  transmissions,  while  another  may  say  little  in  a  long 
time  or  many  transmissions;  without  knowledge  of  the  infor- 
mation content  it  wculd  be  difficult  to  evaluate  these 
situations. 

(6)  Performance  changes.  The  performance  changes  of 
the  vehicle,  desired  as  a  result  of  communicating,  define 
measurement  in  terms  of  links  between  auditory  data  and 
system/mission  performance  data.  For  example,  turning 
performance  can  be  measured  following  a  "hard-as-possible" 
direction  to  the  pilct  of  an  F-4  aircraft. 

2.   Auditory  Data  Processing. 

A  computer-assisted  manual  auditory  processing 
system  is  required  since  automatic  voice  decoding  equipment 
is  not  available.  Expert  personnel  are  therefore  required  to 
identify  complex  performance  and  to  structure  processing 
rules  for  data  clerks  who  reduce  data  to  a  form  allowing 
input  into  a  computer. 
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Three   auditory   data   processing   problems   imposed 
requirements  for  measurement   system  design: 

a.  Identification. 

In  spite  of  normal  communications  clutter,  it  is  necessary 
to  clearly  identify  who  is  talking,  even  if  two  transmis- 
sions are  simultaneously  made  (jamming) .  Voice  key 
circuitry  is  recommended  to  provide  a  digital  signal  indi- 
cating which  crewmenber  is  talking. 

b.  Synchronization. 

Audio  data  must  be  synchronized  with  other  data 
recording  to  permit  relating  auditory  information  tc  corre- 
sponding performance  changes.  Each  recording  device  must 
therefore  include  an  audio  voice  track. 

c.  Data  Reduction. 

Manual  functions  in  audio  data  processing  are 
unavoidable,  but  can  be  minimized  through  computer- 
assistance.  Audio  data  playback  must  be  accompanied  by 
display  cf  performance  parameters,  especially  the  parameter 
of  TIME,  to  allow  computer  correlation  of  manual  audio-data 
entry  with  the  digital  data  base.  A  convenient  means  for 
manual  data  entry  is  a  desirable  feature. 

These  data  processing  problems  must  be 
considered  in  the  design  of  a  combat-crew  training  measure- 
ment system.  Thus,  since  HAF  has  at  most  the  same  aircraft 
as  the  aircraft  of  the  USAF  bases  from  which  data  were 
gathered,  HAF  must  apply  the  same  concepts  for  the  purpose 
of  deriving  combat  crew  performance  measurement.  This 
approach  will  solve  its  problem  of  continuously  optimizing 
the  combat  readiness  cf  its  combat  pilots.  These  concepts 
(measurements)  that  were  discussed  in  this  chapter  are  the 
following: 
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(1)  Measurement  Conironality 

(2)  Common  Measurement 

(3)  Prototype  Measurement 

(4)  Measurement  Parameters 

(5)  Measurement  Descriptions 

(6)  Communication  Analysis 

Of  course,  the  value  of  experience  in  combat  is 
not  easily  determined.  History  says,  experienced  fighter 
pilots  have  done  better  than  inexperienced  pilots  in  combat 
situations.  Intuitive  judgement  says  that  the  greater  the 
experience  level,  the  greater  the  chance  a  unit  has  for 
success. 

Modern  fighters  are  only  as  effective  as  the 
pilots  who  fly  them.  The  HAF  should  continuously  evaluate 
the  need  for  experienced  fighter  pilots  in  the  combat  ready 
units.  The  price  may  be  high,  but  the  price  of  defeat  in  the 
next  battle  for  the  air  may  be  even  greater.  At  the  end  of 
WWI  (World  War  I)  a  dcctrine  was  written,  "...  if  you  hold 
the  air  you  cannot  be  beaten,  if  you  lose  the  air  you  cannot 
win."   [Bef.  28]. 

So  far,  in  order  to  establish  criteria  for 
combat  readiness  of  HAF's  combat  pilots,  we  have  dealt  in 
chapter  II  with  the  principles  of  human  performance,  in 
chapter  III  with  the  criteria  generally  and  their  measure- 
ment, in  chapter  IV  with  the  introduction  of  the  combat- 
ready  crew  performance  measurement,  and  finally  in  chapter  V 
we  derived  the  combat-crew  performance  measurement 
techniques. 
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VI.  COHC10SIONS  AND  RECOMMENDATIONS 

The  Hellenic  Air  Force  has  always  been  interested  in 
keeping  the  highest  level  of  performance  for  its  comtat 
pilots  and  keep  up  with  the  latest  of  technology  achieve- 
ments. Studies,  as  that  of  OSAF,  have  always  been  of  a  great 
concern,  for  the  purpose  that  new  techniques  and  theories 
can  be  revealed  in  its  benefit.  In  this  way,  HAF  keeps  up 
with  the  latest  and  increases  the  efficiency  and  effective- 
ness of  its  combat  pilots.  The  necessity  of  improving 
training  performance  information  of  its  pilots  reguires  new 
methods  of  measuring  flight  crew  performance  during  combat 
crew  training.  As  we  have  seen  in  this  study,  criteria  for 
combat  readiness  of  fighter  pilots  are  : 

(1)  Farametric  referent  or  standard  of  performance. 

(2)  Farametric  limit  about  the  parametric  standard. 

(3)  System  component  criteria. 

(4)  Test  criteria. 

(5)  Multidimensional  in  nature  and  represent  the 
complete  desired  end  result  of  a  system. 

From  the  examination  of  this  and  the  USAF  study  we  can 
conlude  that: 

(1)  Opening  a  new  flow  of  information  can  have  a  major 
influence  on  training  technology.  The  execution  of  effective 
contemporary  training,  the  development  of  new  training 
devices,  and  the  exploitation  of  powerful  concepts  such  as 
adaptive  training,  and  learner-centered  instruction,  all 
depend  on  information  available  through  performance 
measurement. 

(2)  The  role  of  performance  measurement  may  be  that  of 
causal  input,  a  catalyst,  or  a  weak  link  in  a  system  chain, 
but  the  net  effect  of  better  performance  measurement  in  any 
case  is  a  positive  and  possible  revolutionary  improvement. 
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(3)  The  major  difficulty  that  stands  in  the  way  of 
improvements  through  performance  measurement  is  that 
adequate  performance  measurement  systems  do  not  exist  where 
needed.  The  present  study  has  defined  a  guide  for  measure- 
ment in  a  manner  permitting  adaptation  to  specific  needs  and 
budgets. 

(U)  The  status  of  combat-crew  measurement  and  other 
areas  cf  complex  man-machine  performance  is  reflected  by  the 
measurement  analysis.  The  measurement  described  here 
reflects  the  critical  dimensions  of  performance,  and  agrees 
with  the  measurement  structure  used  by  operational  training 
personnel.  Given  this  capability  a  number  of  future 
advances  are  possible.  First,  use  of  this  measurement  will 
lead  to  efficient  interim  and  immediately  available  measure- 
ment for  training.  Second,  improvements  in  the  generation 
of  optimally  efficient  measurement  sets  may  be  expected. 
Third,  clarification  of  the  relationship  between  objective 
and  sub-ective  measurement  should  be  possible.  None  of 
these  results  is  going  to  occur  without  an  acceptable 
performance  measurement  system  and  appropriate 
experimentation. 

This  section  is  concluded  with  the  following  recommenda- 
tions for  future  development  opportunities  of  the  HAF: 

(1)  To  consider  measurement  in  the  context  of  combat- 
crew  training. 

(2)  To  assess  measurement,  that  is  to  identify  potential 
measurement  indicated  by  combat-crew  training  personnel. 

(3)  To  assess  the  constraints  placed  by  the  environment 
on  feasible,  usable  measurement  systems. 

( 4)  To  establish  the  model  of  instructional  system 
development. 

(5)  To  establish  a  measurement  system  for  the  opera- 
tional environment. 
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To  establish  an  automated  measurement  system  that  would 
relieve  the  instructor  pilot,  to  a  maximum  extend,  from  the 
requirements  of  having  to  record  a  great  deal  of  information 
manually  on  the  basis  that  such  activity  degrades  his 
ability  to  competently  instruct  his  student. 

(6)  For  the  purpose  of  deriving  combat-crew 
performance  measurement  to  consider  (a)  measurement 
commonality,  (b)  prototype  measurement,  (c)  measurement 
parameters,  (d)  measurement  specifications,  (e)  communica- 
tion analysis. 

(7)  If  HAF  has  tc  make  a  choice  between  video/photo  and 
digital  recording  approaches  to  measurement  system  design, 
then  the  video/photo  recording  could  be  chosen,  as  a 
performance  measurement  system,  because  of  its  low  cost, 
information  provided,  flexibility  and  ease  of  use.  A  hybrid 
system  combining  the  advantages  of  both  is  preferable  to 
either  type  of  recording  alone.  Due  primarily  to  cost,  the 
bulk  of  measurement  parameters  would  be  derived  from  a 
video/photo  system,  and  the  remainder  with  a  small  digital 
recording  capability. 
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APPENDIX  A 
PECTOTYPE  MEASUREMENT 

In  this  Appendix  an  example  of  prototype  measurement  is 
presented  for  takeoff  and  climb.  The  format  used  is  to 
present  a  discussion  together  with  prototype  measurement, 
indicating  through  the  Figure  A.1  the  types  of  information 
which  are  considered  important  to  a  description  of  pilot 
performance. 

TAKEOFF  &   CLIMB 

All  aircraft  takeoff  and  climb  to  a  cruising  altitude 
and  configuration.  Fixed-wing  aircraft  perform  these  maneu- 
vers in  tasically  the  same  way;  however,  at  a  detailed  level 
there  are  distinct  differences  between  aircraft.  Thus, 
measurement  must  be  tailored  to  each  aircraft,  but  the 
general  structure  of  such  measurement  may  be  defined  so  that 
the  essential  elements  are  constant  across  aircraft.  The 
following  sequence  is  rather  basic:  Takeoff  roll,  Rotation, 
Liftoff,  Gear-up,  Flaps-up,  Climb  and  Level-off.  The  infor- 
mation desired  within  each  of  these  flight  maneuvers  may 
also  be  expressed  in  a  substantially  common  manner. 

Conditions.  To  properly  interpret  measurements  made 
during  a  particular  flight,  information  on  the  conditions 
existing  at  the  time  are  needed.  The  gross  weight,  wind 
direction  and  velocity,  runway  direction  and  length, 
temperature,  altimeter  setting,  field  elevation,  and  posi- 
tion of  the  aircraft  in  formation,  are  reference  data  for 
the  evaluation  of  performance. 

Takeoff  roll.  The  takeoff  will  be  assumed  to  begin  with 
the  application  of  power.  The  takeoff  roll  maneuver  will  be 
considered   finished  at   rotation.    The   objective   is  to 
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TAKEOFF    S    CLIMB* 

CONDITIONS: 
"Cross  WZ:_                  fling:_   _/            Runway:      _/_ 
Temp.: Alt.  Set.: "Field   Elev. Form   Pos.  : 

TAKEOFF    EOLL:     (To    power  until   rotation) 

"Power   Sel:~  TentefTme   Uev.  :    Min,    Max,    Av. 

Reject    Speed":    Computed  Headinq:Min,    Max,    Av. 

Time: Dist: Bank:    8    Max,    L   Max 

ROTATION:     (Nose   g.ear   off  until  p_itch   att.    established) 

Eo^. "Speed: 2_  ~Sfa£.    Trim: ~* 

Pitch:      Rate: Bank: 

Final:" CenterIine~tJev.  :_ 

Overshoot:  Heading: ~_ 

LIFTOFF:     (Pos.    Vert.    Vel.) 
Iirspeed:_                           "Pitch:  Bank:_  Hdg: 

Vert.    Vel.    Ifter: Sec: 

GEAR-DP:     (Handle    up   until  gear-up   5   locked) 
~G"ear~TJp   SpeeaTI_I  V7T7"TniT77~_  V.VTTTnal: 
Pitch: Bank: Hdg: 

FIAIS   UP:    (Start    up   to    full   up)    ** 

TrimTJII                                          l=5Z^)nIy  IAS   PITCH    ALT  VV  TRIM 

Pitch:        Bank:         Hdg:             Start             x           x          x  x  x 

A/S    (IfllT)            TFINAL)"            1st    Pos         x           x           x  x  x 

VV    (INIT)               (FINAL)"           2nd    Pos        x           x          x  x  x 

ALT     (INIT7~__       (FINAL)"        Full                x           x           x  x  x 

CLIMB    &    LEVEL    OFF:     ( Deepen  ds   on    Flight    Plan) 

INIT    FINAL 

PER    A/S  MACH    HDG    ALT    ALT    PITCH    TRIM 

Accelerate                             x        x  x        x"      x        x        "x             x 
Climb   A/S       (#1) 

(#2)                  xx  xxxx           x             x 

Climb    MACH                              xx  xxxx           x             x 
Level-off    (Alt-10*   VV) 

(to    Cruise)    xx  xxxx           x             x 


*  Also,  mandatory  communication  &  instances  where  A/C 

limits  are  exceeded. 
**F-106  has  no  flaps 


Figure  A. 1    Example  of  Prototype  Measurement 
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accelerate  in  a  straight  line  along  the  centerline,  or 
parallel  to  the  centerline,  with  wings  level.  Power  and 
resultant  acceleration  must  be  checked;  for  heavy  aircraft 
and/or  short  field  takeoffs,  accelaration  checks  are 
formally  performed.  lime  and  distance  along  the  runway  are 
checked  against  airspeed  to  determine  if  necessary  accelera- 
tion performance  is  lacking  in  the  time  to  safely  stop  the 
aircraft.  Reject  speed  is  noted  in  case  of  an  emergence.  The 
formation  flight  leader  must  slightly  reduce  power  to  allow 
a  margin  of  thrust  control  for  other  members  of  the  flight. 

Rotation.  Proper  rotation  is  normally  necessary  to 
achieve  predicted  takeoff  performance.  Rotation  will  be 
defined  as  the  activities  between  the  time  that  the  nose 
gear  lifts  off  the  runway  until  the  time  that  a  stable  pitch 
attitude  is  established.  Stabilizer  trim  is  important,  bank 
angle,  centerline  and  heading  deviations  should  be  small. 
Rotation  should  occur  within  1-2  KIAS  of  the  desired  rota- 
tion speed.  The  rate  of  rotation  should  not  be  either  too 
large  or  too  small.  A  specific  pitch  attitude  should  be 
established  witout  overshoot  or  oscillation. 

liftoff.  Liftoff  is  a  dicrete  event,  occurring  when 
vertical  velocity  is  positive.  At  this  time,  the  airspeed, 
pitch  angle,  bank  angle,  and  heading  are  noteworthy.  The 
vertical  velocity  a  short  time  after  liftoff  may  also  be 
measured  to  indicate  whether  the  aircraft  is  positively 
airborne,  or  if  there  is  any  tendency  to  settle  back  to  the 
runway. 

£§ar-U£.  Measurement  should  be  taken  from  the  time  that 
the  gear  handle  is  raised  until  the  time  that  the  landing 
gear  are  up  and  locked.  The  initial  speed  at  which  the  gear 
are  raised,  the  change  in  vertical  velocity  during  the  time 
that  the  gear  are  coming  up,  and  pitch,  bank,  and  heading, 
should  be  measured. 
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Flaps-up.  Flaps-up  measurement  is  treated  in  somewhat 
the  same  manner  as  for  gear-up,  for  the  tasks  are  somewhat 
the  same;  a  configuration  change  is  occurring  which 
presents  a  perturbation  in  longitudinal  control.  A  trim 
change  occurs,  and  pitch,  bank,  and  heading  must  be 
controlled.  Normally,  flaps  must  not  be  raised  before  a 
specific  altitude  and  airspeed  (but  before  maximum  flaps 
speed),  and  during  the  transition  to  flaps-up,  changes  in 
airspeed,  vertical  velocity,  and  altitude  indicate  whether 
the  maneuver  is  properly  performed. 

The  B-52  presents  a  special  measurement  requirement 
since  a  specific  speed  schedule  must  be  maintained  as  flaps 
are  raised;  in  addition  to  airspeed,  pitch  angle,  altitude, 
vertical  velocity,  and  stabilizer  trim  are  of  interest 
during  this  period  of  time. 

Climb  and  Level-off.  For  each  aircraft,  there  are  a 
number  cf  methods  for  climb-out  depending  on  the  flight 
plan,  and  desires  fcr  economy  or  performance.  It  may  be 
desirable  to  measure  climb  performance  from  liftoff,  cr  to 
start  when  the  aircraft  is  in  a  clean  configuration.  This 
phase  may  be  divided  into  the  following  parts:  acceleration, 
maintain  climb  airspeed  (may  be  several  increases  in 
airspeed  during  the  climb) ,  maintain  climb  Mach  number,  and 
level-off  (normally  level-off  begins  at  an  altitude  which  is 
below  cruise  altitude  by  10%  of  the  vertical  velocity). 
Power,  airspeed,  Mach,  heading,  initial  and  final  altitude, 
pitch  angle,  and  trim,  are  parameters  which  may  be  measured 
during  each  portion  of  climbout. 


63 


IIST  OF  REFERENCES 


1.  Wright-Patterson  AFB  Ohio:  Aerospace  Medical  Research 
Laboratories,  Report  AMRL-TDR  62-2,  The  Measurement  of 
Advanced  Flight  vehicle  Crew  Prof lcieccy  in  SynTFetlc 
tfrouncPEnvironments,  By  XTE.  SmocTe,  A.'  Srubef ,  an"3 
J7H7-Ely7~T?F2:: 

2.  Nelson,  W.A.,  Knott,  T.A.  and  Carhart,  P.W.,  Webster*  s 
New  International  Dictionary  of  the  English  language, 
HeconcFecTiTTon ,  unabridged",  G.~  5  C7  "Herriam  Company, 
Publishers    Springfield,    Mass.,    U.S.A.,    1957. 

3.  The  RAND  Corporation,  Report  1435-PR,  l2LP.E2.ZiH3  SAC 
Aircrew  and  Aircraft  Scheduling  To  Increase  Resource 
Er"fec:£Tveness#    By  H7H.  "Herman,    p.    II X, "July   T97^T. 

4.  The  RAND  Corporation,  Report  547-PR,  Force  Capability 
Reporting,  by  C.F.  Bell,  T.T.  Tierney,  p."  "23, 
SejffemBer    1970. 

5.  Brooks  AFB,  Texas:  Aerospace  Medical  Research 
Laboratory,  Report  AMR1-TR-67-97 ,  Development  and 
Evaluation  of  a  Digital  Computer  Program  lor  luToEaTTc 
"Human  ""Performance  Honif oring  in  ITigJII-  Simulator 
Training.,    "^Y   P«  A.    Knoop,    A"ugust    19T8. 

6.  Brooks  AFB,  Texas:  Air  Force  Medical  Research 
Laboratory.  Report  AMRL-MP-4  0,  Considerations  in  the 
Design  of  Automatic  Proficiency,  fleasuremenT  Hguipmen^ 
"in  Simulators,  By  R.  TucTKout^,  and"  TTU.  Cot^erman, 
June~T9S3T 

7.  Ft  Worth,  Texas:  U.S.  Naval  Training  Device  Center, 
Life  Sciences,  Inc.  ,  Report  No.  NAVTRADEVCEN-TF. 
1388-1,  An  Experimental  Program  for  Relating  Transfer 
of  Training  tc  "PITo^  "Performance  anci  Degree  or 
Simulation,  by~P.  G.-~T3emaree,~  "DTI."  Norman,  and  W.I?. 
HaTKeny,    Uune    1965. 

8.  Mclean,  Virginia:  Quest  Research  Corp.,  Report 
AFERL-TR-74-88,  Candidate  T-37  Pilot  Performance 
Measures  for  Five  CoiTEacf~laneuvers,  ~By~E.  M .  TonneUy 
and   otners,    December    1  y  /"?. 

9.  Tempe,  Arizona:  Arizona  State  University,  Tempe 
Department  of  Educational  Technology,  Report  TR-40830. 
Measurement  of  Flight  Performance  in  a  Flight 
Simulator,  by  B.U.  "SEipIey,  ~T7S.  Gerlak  and*  F.H. 
Breck"e,    lugust    1974. 

10.  Krendel,  E.,  Eloom,  J.,  The  Natural  Pilot  Model  for 
Flight  Proficiency  Evaluation,  FranklTn  InstiTucTe, 
P£Tladelph~ia7~T":fi±    19  637 

64, 


11.  Pecsacola,  Florida:  Naval  Aerospace  Medical  Institute, 
Naval  Aerospace  Medical  Research  Laboratory,  Report 
1158,  A  New  Approach  to  Criterion  Development  in  the 
Replacement  Air  Sroup,— Ey  R~.IT.  SKannon,  F.  LT  TJaag,  anH 
JTcT   Fergurson,   July    1972. 


12.         McCormick,      E.J.,        Sanders,      M.S.,       Human      Factors   in 

~di~Eio 
1982. 


Enqinerinq      and      Design,         Fifth         Edition,         p."    "257 
flcgraw-HTTl   Bock  "Company,    New    York, 


13.  Giltert  T.,  Human  Competence:  Engineering  Worthy 
Performance,    p.    25,    McGiraw    HlIT,    New    York",    7"3/8.~ 

14.  Bailey,  R.W.,  Human  performance.  Engineering^:  A  Guide 
for  System  Designers,  "p.  ~72T,  P"r ehTice-H'all,  "Tnc. 
Englewoo3~CliIIs,~TTew    Jersey    07632,    1982. 

15.  Salvendry,  G.  and  Seymour,  W.D.,  Prediction  and 
Development  of  Industrial  Work  Performance, ~~p.~  1"S3, 
Ton~n   Wiley    &Sons,    lnc7~TTew   Yof k,~T913~. 

16.  Ghiselli  E.E.,  The  Validity,  of  Occupational  Aptitude 
Tests,    Willey,    LoncTon,    T'SBT. 

17.  Salvendry,  G.,  "Learning  Fundamental  Skills-  A  Promise 
for  the  Future,"  AIIE  Transactions  on  Industrial 
Engineering  Research"  and  Be velopmenf ,  v.  "17  pp. 
3TJ^=3U57^Tf697 

18.  Passmore,  R.  ,  Durinf  J.V.G.A.,  "Human  Energy 
Expenditure,"  Physiological  Reviews,  v.  35,  p. 801-840, 
1955.  — 

19.  Salvendry,  G-,  Pilitsisf  J.,  "Medical  Suturing-  The 
Acquisition  and  Training  of  Related  Psychomotor 
Skills,"    Ergonomics,    v.     14,    pp.    703-711,    1973. 

20.  Rodger,  A.,  "The  Criterion  Problem  in  Selection  and 
Guidance."  Occupational  Psychology,  v.  39,  Number  2, 
pp.    77-82,    795"5T 

21.  Barnes,  R.M.,  Motion  and  Time  Study,  6th  ed.,  John 
Willey,    New   York7~T9F8". 

22.  Obermayer,  W.E. ,  Vreuls,  D. ,  Combat-Ready  Crew 
Performance  Measurement  System,  Williams  A~FB,  Arizona, 
D"ec~TT7?7 

23.  Department  of  the  Air  Force,  Headquarters,  USAF 
Tactical  Fighter  Weapons  Center,  TAC.  TEST  69^4F, 
Audio-Video  Recording  System  (AVRSj",  Nevada,  ~Z3 
5epTemT5er    T97TT. 

24.  Department  of  the  Air  Force,  Headquarters,  U.S.  Air 
Force,  A.F.  Manual  50-2,  Instructional  System 
Development,    Washington,    D.C.  ,    31    ITec   7T7U. 

65 


25.  Department  of  the  Air  Force,  Headquarters,  USAF 
Tactical  Fighter  Weapons  Center,  Report 
1AC-TR-70A-1 13F,  A-7  Airborne  Video  Recording  System 
(MRS)  /  Nevada,  FeE~797T.  ~ 

26.  Williams  AFB  AZ.:  Flying  Training  Division  Air  Force 
Human  Resources  Laboratory,  Report  AFHRL-TR  72-55, 
Evaluation  of  Airborne  Audio- Video  Recording  as  a  Tcol 
for  Training  m  tne  I-7TJ  Taclical  Tighter .  by  U7A". 
Fi^zeralcl  and  DTI.  "HouTTon,  Uc"t"oT5er  TT7T." 

27.  United  States  Air  Force  Systems  Command  Manual,  AFSMC 
375-5,  Systems  Engineering  Management  Procedures ,  10 
March  196b. 

28.  Blanch,  C.C.,  Air  Superiority  Today,  and  Tomorrow,  p. 
15,  Air  University,  TlaxweTI  IFB,  Ilabama,  IprTI  1916. 


66 


INITIAL  DISTRIBUTION  LIST 


No.  Copies 


1.  Defense  Technical  Information  Center  2 
Cameron  Station 

Alexandria,  Virginia  22314 

2.  Library,  Code  0  142  2 
Naval  Postgraduate  School 

Monterey,  California  93943 

3.  Willis  E.  Greer  Jr.,  Code  54Glc  1 
Department  of  Administrative  Science 

Naval  Postgraduate  School 
Monterey,  California  93943 

4.  Professor  Douglas  E.  Neil,  Code  55Ni  1 
Naval  Postgraduate  School 

Monterey,  California  93943 

5.  CDE  Charles  Willis  Hutchins  Jr.,  Code  55Hu        1 
Naval  Postgraduate  School 

Monterey,  California  93943 

6.  Cpt  Anestis  Hatzipouflis  3 
Egnatias  116,Pilea,  Thessaloniki,  Hellas 

7.  Mr.  Dimitrios  Hatzipouflis  1 
Egnatias  116,  Pilea,  Thessaloniki,  Hellas 

8.  Mr.  Apostolos  Karabourniotis  1 
Chrissostomou  Smirnis  2,  Kavalla,  Hellas 


67 


1  3  5  3  T 


210235 


Thesis 

H327 
c.l 


Hatzipouflis 

C  r  i  t  e  r  la^^oT__comb  a  t 
rea-etTness   of  combat 
)ilots. 


